Accelerated Landweber methods based 
on co-dilated orthogonal polynomials 

Wolfgang Erb * 
18.12.2012 

In this article, we introduce and study accelerated Landweber methods for 
linear ill-posed problems obtained by an alteration of the coefficients in the 
three-term recurrence relation of the z^-methods. The residual polynomials 
of the semi-iterative methods under consideration are linked to a family of 
co-dilated ultraspherical polynomials. This connection makes it possible to 
increase the decay of the residual polynomials at the origin by means of 
a dilation parameter. This increased decay has advantages when solving 
linear ill-posed equations in which the spectrum of the involved operators 
is clustered at the origin. The convergence order of the new semi-iterative 
methods turns out to be the same as the convergence order of the original 
j^-methods. The new algorithms are tested numerically and a simple adaptive 
scheme is developed in which an optimal dilation parameter is computed. 
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1 Introduction 



The aim of this article is to present and to investigate specific accelerated Landweber 
schemes that constitute an alternative to the well-known i/-methods and which, depend- 
ing on the given data, are able to display an improved performance. For the necessary 
notation, we give first a short summary about linear ill-posed problems, the Landweber 
iteration and semi-iterative methods. The theoretical background on ill-posed problems 
and their numerical solution is mainly taken from the monographs |10j and [30] . Further 
introductions can be found in |14^ [22| [23| [25] and the references therein. 
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1 Introduction 



In the Hilbert space setting of linear ill-posed problems, one considers a bounded linear 
operator A : Tii — )■ between two Hilbert spaces Hi and and the solutions of the 
linear equation 

Af = g, f eUi, g<-n{A). (1) 

If the range TZ{A) of A is not a closed subspace of 7^2) the solution / of ([TJ does not 
depend continuously on the initial data g and the linear system ([TJ is called ill-posed. 
To solve such ill-posed equations, Landweber suggested in |24| the iterative scheme 

fn+i = fn + 2ujA*{g- Afr,), n > 0, (2) 

to compute the minimum norm solution of the normal equation A*Af = A*g. Here, the 
operator A* : T-L2 — )• T^i denotes the adjoint of A and a; > is a relaxation parameter. 

For the initial vector /o = 0, the Landweber iterate fn+i belongs to the Krylov space 

)Cn{A*A, A*g) := {A*g, {A*A)A*g, {A*AfA*g, • • • , {A*ArA*g} C ^i. 
and can be expanded as 

fn+i=Pn{ujA*A)ujA*g with Pn{y) = ^ • (3) 

y 

Then, for the minimum norm solution / of A*Af = A*g, we get (cf. |10| formula (6.12)]): 

/-/n+i=r„,+iMM)/ with r„+i(y) = (l-2y)"+^ (4) 

We remark, that in contrast to other references we use an additional scaling factor 2 
in the definitions ([3|) and @ of the Landweber polynomials p„ and r„. This particular 
scaling ensures that the residual polynomials r„(y) converge pointwise to zero precisely 
in the interval ]0, 1[. So, if Ct;||A*A|| < 1 holds, the spectral decomposition of the positive 
semidefinite operator A* A ensures the convergence of the iterate /„ to / (see |10| Theorem 
4.1 and Theorem 6.1]). 

A disadvantage of the Landweber scheme is the slow convergence of r„(y) = (1 — 2y)"' to 
zero if y is close to zero. To circumvent this drawback, it is favorable to substitute the 
polynomials Pn and r„ of the Landweber iteration with more suitable ones. These more 
sophisticated iteration schemes are commonly known as semi-iterative or accelerated 
Landweber methods (see [9], Chapter 6], [IB], [IT], [301 Section 5.2] and [32]). In 
principle, in formula ([3|) one could use every sequence pn of polynomials with exact degree 
n as a semi-iterative method. However, in order to evaluate the iteration polynomials Pn 
and the respective residual polynomials r„+i(y) := 1 — ypn{y) in a cost-effective way, it 
is advantageous to use sequences of orthogonal polynomials (see |16|). 

If Pn{x), n G No, denote monic polynomials of degree n orthogonal with respect to a 
weight function w supported on the reference interval [—1, 1], the polynomials P„ can be 
evaluated cheaply by the three-term recurrence relation (cf. [SJ I. Theorem 4.1]) 

Pn+l{x) = [X - an)Pn{x) - (inPn-l{x), Pq{x) = I, Pi{x) = X - ttQ. (5) 
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It is well-known that the coefficients a„ G M and /3„ > are uniquely given and that 
P„(l) > holds for all n G Nq. Now, if we define the residual polynomials r„ on [0, 1] by 

the constraint r„(0) = 1 is satisfied and ([5]) yields the following recurrence formula: 

rn+i{y) = (1 - a. - 2y)^^r^{y) - Pn ^p'\]^ p^"^!! / n-i(y), n > 1, (7) 

My) = 1, = 1 - -r^ — y- 

1 - "0 

Also the coefficients fin+i = p^^^/(i) can be computed recursively via ([5]) as 

^'n(l) Pn(l) 1 



P„+i(l) (l-a„)P„(l) -/3„P„_i(l) (l-a„) 



The resultant recursion formula for the iteration polynomials Pn{x) = yields 
the following semi-iterative algorithm (stated with a slightly different notation in |16j): 

Algorithm 1 Semi- iterative method based on monic orthogonal polynomials on [—1, 1] 
/^i = rr^ 

fo = 0, /i = 2/ii ujA*g 

while (stopping criterion false) do 
,, _ 1 

fn+l = fn + ((1 - an)/in+l " l)(/n " fn-l) + 2^n+l U}A*{g - Afn) 

71 — 7- n -|- 1 
end while 



Setting cXn = (3n = 0, Algorithm [T] describes the Landweber iteration ([2]). Other well- 
known examples of Algorithm [T] are based on the Chebyshev polynomials C/„ of the second 
kind and the Jacobi polynomials pji' v > 0. In the first case, the scheme is 

known as Chebyshev method of Stiefel, in the second case as the i/-methods of Brakhage 
[3]. For v = 1, the scheme is known as Chebyshev method of Nemirovskii and Polyak 
(see |28|). We remark that in this article the parameter v of the z^-methods is set twice 
as large as normally used in the literature. In this way, the parameter ly coincides with 
the parameter of the ultraspherical polynomials. As a stopping criterion for Algorithm 
[H several choices are possible (see |16|). However, the most common one is certainly the 
discrepancy principle of Morozov and some generalizations of it. 

To analyse the convergence of Algorithm [H we consider smooth solutions / of ([TJ in sub- 
spaces Xg := TZ{{A* A)^), s > 0, of the Hilbert space Hi and the moduli of convergence 

es{n) = sup \y^rn{y)\, ef («) = sup |y^(l - 2/)5r„(y)|, n G N. 
?/G[0,l] ye[o,i] 
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We will use the modulus £s{ti) if the residual polynomial r„(y) converges to zero at y = 1 
and ef(n) otherwise. In the first case, if / = {u!A*A)2h E Xg holds with h S Tii and 
< 1, the spectral theorem yields the error estimate (see [16| Theorem 3.2]) 

||/-/n|| <es(n)||/i||. 

In the second case, we assume that a;||y4*y4|| < 1 holds such that id -co A* A is an invertible 
operator on Tii- Then, a similar argumentation as in |16[ Theorem 3.2] yields the bound 

II/- /nil < 



{l-uj\\A*A\ 



The convergence rate of the Landweber method is known to be of order ef (n) = 0(n 2 )j 
while the z^-methods reveal es{n) = 0(n~*) for < s < v (see \10\ Chapter 6].|16|). 



The main goal of this article is to find and investigate orthogonal polynomials P„ with 
a priori given recurrence coefficients a„ and /3„ such that the resulting semi-iterative 
scheme in Algorithm [1] improves the performance of the z/-methods. More precisely, 
under some general assumptions on the given data A and g, we want to determine new 
semi- iterative methods in which the error \\Afn — g\\ in Algorithm [1] is smaller compared 
to the z^-methods. If Algorithm [1] is stopped according to the discrepancy principle, this 
will result in an earlier termination of the iteration. However, a well-known result (cf. 

Theorem 4.1], |17) ) states that the convergence order es(n) = 0(n~*), < s < v, 
of the z/-methods is already optimal and that it is not possible to obtain semi-iterative 
methods with a better order. 

Searching for new semi-iterative schemes, we focus therefore on a different important as- 
pect: the decay of the residual polynomials r„(?/) at ?/ = 0. For linear ill-posed problems, 
the operator A* A on Tii is typically compact and its spectrum is clustered at the origin 
y = 0. Thus, if we assume that most of the spectrum of A* A is concentrated at y = 0, 
the error — in Algorithm [1] depends strongly on how fast the residual polynomials 
fniu) decay to zero in the neighborhood of y = 0. For this reason, various concepts of 
fast decaying polynomials have already been studied, see |17| and the references therein. 

If the residual polynomials are orthogonal, the decay of rn{y) at y = is directly 
linked to the location of the smallest root of r„(y) in the interval [0,1]. The closer to 
the origin the smallest root is, the faster r„(y) decays at y = 0. This link is now used 
to construct polynomials r„ with a faster decay at y = than the residual polynomials 
of the z^-methods. To this end, we alter particular coefficients in the recurrence relation 
of the orthogonal polynomials linked to the z^-methods. This altering leads directly to a 
family of co-dilated orthogonal polynomials, many of whose characteristics are known in 
the literature, see [Hj [20 ] [26 l I3 H |33]. Based on these co-dilated orthogonal polynomials, 
we will construct the new semi-iterative methods and investigate some of their properties. 

The main idea of this article is explained in more detail in the next section on the ba- 
sis of the Chebyshev polynomials. In Section 3, the theoretical fundamentals of the 
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2 Co-dilated Chehyshev polynomials 



co-dilated orthogonal polynomials are laid and some properties of their extremal roots 
are investigated. Section 4 is devoted to the particular family of co-dilated ultraspher- 
ical polynomials and their properties. In Section 5, the transition from the co-dilated 
ultraspherical polynomials to the co-dilated i^-methods is illustrated. The main results 
of Section 4 and 5, formulated in Theorem 14.41 and Corollary 15. 2| state that under cer- 
tain conditions on the dilation parameter the convergence order of the new semi-iterative 
methods is the same as for the i^-methods. In Section 6, it is shown how for the co-dilated 
1-method the dilation parameter can be fitted optimally to minimize the error — 
Finally, in the last section some numerical tests are conducted. 

In this article, the coefficients an and /3„ in Algorithm [1] are always a priori given. 
Disabling this constraint, a powerful alternative is given by the method of conjugate 
gradients where an and /3„ depend on A and g (see |1U| Chapter 7], |12| . |30| Section 5.3]). 
It is well-known that the iterate /„ of the cg-algorithm minimizes the error \\Afn — g\\ in 
the Krylov space ICn-i{A* A, A*g). On the other hand, the cg-iteration has a multifarious 
convergence behavior that makes it harder to handle as a regularization tool than the 
z^-methods. A deep analysis of the cg-algorithm as a regularization tool and a comparison 
with the z/- methods can be found in [101 Chapter 7], [16] and |30l Section 5.3]. 



2 Co-dilated Chebyshev polynomials 



We illustrate the idea of this article on the basis of the Chebyshev polynomials of the 
second kind. For x = cost, t € [0, vr], the monic Chebyshev polynomials of the first and 
the second kind are explicitly given as (cf. \1'6\ p. 28]) 

m / \ 1 rr / \ 1 Slufn -|- l)t 

r„ cost = rcosnt, t/„ cost = . 

^ ^ 2"-i ' ^ sint 

Further, we consider linear combinations of C/„ and T„, i.e. 

U:ix) = U*n{x, A) := (2 - X)Un{x) + (A - l)r„(x) (8) 

= \Un{x) + {l-X)xUn-l{x), 

= Unix) + i^C/„_2(x), A G R. 

The last two identities in ([8]) follow from simple trigonometric conversions. In order 
to use these polynomials in a semi- iterative scheme, we introduce according to ([6]) the 
residual polynomials 

<(y) = <(y,A) = ^^i__lZ, ye [0,1]. 

In Figure[Tl the normalized polynomials Uq{x)/Uq{1), Tq{x) /Tq{1) and C/|(x)/C/|(l) with 
A = 1.5 are plotted. For —1 < a; < 1, the polynomials C/„(a;)/J7„(l) converge pointwise 
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U^ix)/Uiil),X = 1.5. 



Figure 1: The Chebyshev polynomials Uq/Uq{1), Tq/Tq{1) and the linear combination 
C/gVC/g (1), A = 1.5, on the interval [-1, 1]. 



to zero as n — )• oo. Thus, also the residual polynomials 1) converge pointwise to 

zero for y g]0, 1[. They form a convergent semi-iterative scheme, the so-called Chebyshev 
method of Stiefel (cf. |30l p. 116]). On the other hand, the polynomials r„(x)/r„(l) do 
not converge pointwise to zero on [—1,1]. Nevertheless, the largest root of r„ is much 
closer to X = 1 than the corresponding root of the polynomial Un ■ This implies that the 
smallest root of the respective residual polynomial r^{y,2) is closer to y = and that 
r* 2) decays faster at y = than the polynomial r* (y, 1). 

Thus, although not giving a convergent iterative scheme, the residual polynomial r* (y, 2) 
has the favorable property to decay fast at the origin. In order to combine both requests, 
a convergent scheme and a fast decay at y = 0, we consider now the linear combinations 
U* of Un and T^. With the identities (see ^ Section A.2, A. 3]) 



Un{l) 



n 



+ 1 , , 1 , , , n(n + l)(n + 2) , , , 

— , TJi) = — T, u'n) = ^ ^ ^ \ T 1 



n 



(9) 



2^ ' "\ / 1 ' ~ n\ I 2713 ' ' 2"~1 ' 

we get by a simple computation the following formula for the derivative of [/* at x = 1: 



C/*'(l) (2-A)C/4(l) + (A-l)r^(l) 



(2 - A)C/„(1) + (A 
(2 



l)rn(l) 



^^ n(„+l)(n+2) ^2(A 



n 



+ 2 2 \{r? - 1) 



(2-A)(n + l) + 2(A-l) 



3 (2 - A)n + A 



Hence, for A < 2, ?7*'(1)/J7*(1) is an increasing function of the parameter A. Therefore, 
also the decay r*'(0) = — C/*'(l)/C/*(l) of the residual polynomials at y = gets faster 
with increasing A. So, we can conclude that for 1 < A < 2 the residual polynomials 
r*(y. A) have a faster decay at y = than the residual polynomials r*(y, 1) of the 
Chebyshev method. On the other hand, it is visible in Figure [1] that the oscillations of 
the polynomial U*{x), A = 1.5, in the interval [—1, 1] have a larger amplitude compared 
to the polynomial Un{x). This holds generally for 1 < A < 2 and is also visible in the 
following convergence result. 
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Theorem 2.1. For X < 2, x € [—1,1], the polynomial U*{x), n > 3, is bounded by 



< 



1 + |1- A| 



2- A i+7r^^(n-l)' 



(10) 



For the residual polynomial r^{y) = ^"jjl/i)^^ on [0, 1] and the modulus of 



convergence 



£i{n), we get the estimates 



K{y)\ < 



1 + |1-A| 



2-A i + 7^(r3^(n-l)' 



£^ n) < 



1 + |1-A| 1 
2 - A n- r 



fll) 



Proof. By the third identity in ([8|), we get for A < 2 and x = cost, t G [0,7r], the bound 



x2 + 



n + 1 



< sin t + 



1 



n + 1 



sin((n.+l)t) 
s'mt 



+ |1- A| 



sin((n— l)t) 
s'mt 



(2 - A)n + A 



^ 2(1 + |1-A|) ^ 2(1 + |1 - A|) 



(2 - A)(n - 1) + 2 - (2 - A)(n - 1) ■ 
Dividing both sides by (V^ — x'^ + ) can conclude for n > 3: 



C/*(x) 



< 



2(1 + |1-A|) 



(2-A)(n-l)(yr^+^, 



< 



1 + |1- A| 



2- A i+(n-l)Vr 



x^ 



The estimates for the residual polynomials r*(y) and the modulus ef (n) follow immedi- 
ately from the estimate of U* (x) . □ 



Theorem 12.11 states that for all A < 2 the symmetric modulus ef (n) has the same order 
0(n~^) of convergence. However, the factor ^'^2^'^^ (|lip has a considerable impact 
on the error estimates if A is close to 2. If A = 2, the polynomials U* correspond to the 
Chebyshev polynomials T„ of the first kind and the corresponding semi-iterative scheme 
is not convergent. 

We have seen so far that for 1 < A < 2 the residual polynomials r*(y,A) decay faster 
at y = but implicate slightly larger error bounds in Theorem 12.11 than the residual 
polynomials r*(?/,l) linked to the Chebyshev polynomials It depends now on the 
given operator A and the right hand side g, whether it is favorable to choose C/„ or U* in 
a semi- iterative scheme. Assuming that most of the spectrum of A* A is concentrated at 
y = 0, the choice of U* in Algorithm [1] with an appropriate A > 1 can have advantages 
compared to Un- 
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Finally, we derive the recurrence coefficients of the semi-iterative scheme based on the 
polynomials U*. For the polynomials U*, we have first the three-term recurrence relation 

U:+,{x) = xU:ix)-\u*_,i^), n>2 (12) 
USix) = 1, U^{x) = X, U^{x) =x^- X^. 



It is well-known that the monic polynomials r„ and C/„ satisfy (|12p with A = 2 and 
A = 1. Then, it follows immediately that (jl2p holds for the linear combination U*. 



In view of (fT2|) . the polynomials U* turn out to be a particular family of co-dilated 
orthogonal polynomials constructed by dilating a coefficient in the recurrence relation 
of the polynomials [/„ by a factor A. This special construction and the consequences 
regarding the roots of U* are investigated in more detail in the next section. In the 
following, the polynomials U* are referred to as co-dilated Chebyshev polynomials. 

The coefficients f^n+i in Algorithm [1] can also be computed explicitly as 

U*{1) f2-A)(n + l) + 2A-2 (2-X)n + X 



C/*+i(l) (2-A)(n + 2) + 2A-2 (2-A)n + 2' 



Therefore, using the recurrence coefficients of the polynomials U* in Algorithm [H we get 
the following recurrence formula for the iterates: 

/i = 2lo A*g, fo = 0. 
For A = 1, this iteration corresponds precisely with the Chebyshev method of Stiefel (see 

m p- 116]). 



3 Symmetric co-dilated orthogonal polynomials 

In this section, we generalize the concept of the co-dilated Chebyshev polynomials to 
arbitrary symmetric orthogonal polynomials on the interval [—1,1]. We denote by P„ 
the monic polynomials of degree n orthogonal with respect to an axisymmetric weight 
function w supported on [—1,1]. In this case, the coefficients a^, ?t- € Nq, in ([5]) vanish 
and we obtain the three-term recurrence relation 

P„+i(x) = xP„(x) -/3„P„_i(x), Poix) = l, Piix) = x, (13) 

with positive coefficients /3„ > 0, n G N. The monic co-dilated orthogonal polynomials 
Pnix) = P^{x,X,m) are now derived from the original polynomials P„ on [—1,1] by 
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dilating the coefficient in the three-term recurrence relation by a factor A € M. 

Po*(x) = i, pr(x) = x, 

P:+,{x)=xP:{x)-PnP:-i{x), n^m, (14) 

If A > 0, Favards Theorem ensures that P*{x), n G N, is a family of orthogonal polyno- 
mials. For m = 1, the co-dilated orthogonal polynomials Pn{x) were firstly introduced in 
[7] by Dini and then generalized in [HJ [31] . Many properties of the zeros of the co-dilated 
orthogonal polynomials like interlacing behavior and the distribution of the zeros are 
well-known and studied in [20) . [26] and [33]. We will add some more properties in the 
course of this section. 

First of all, the co-dilated polynomials can be represented with help of the numerator 
polynomials associated to P„ (see [26|). Therefore, we denote by Pli^\x) the m-th. 
numerator polynomials of Pn defined by the shifted recursion formula 

Pi";:lix)=xPt\x)-bn+rnPt\{x), U > 1, (15) 



Pt\x) = l, Pt\x) 



X. 



Then, by a simple induction argument, the co-dilated polynomials P* can be written as 

P*{x) = \Pn{x) + {l-\)Pm{x)P^rl^^n^{x), fom > m, (16) 
P^(x) = -Pn(a^), for n < m. 



Now, we investigate the behavior of the zeros of P* if the dilation parameter A in the 
recurrence relation (jl4p is altered. We denote by Xnj and 1 < j < n, the n zeros of 
Pn and P* in ascending order. Then, we get the following result for the extremal roots 
of P* and Pn. 

Theorem 3.1. The largest zero Xnn ^^f Pn{x) is a monotone increasing function of the 
dilation parameter X, the smallest zero x* is a monotone decreasing function of X. In 
particular, for A > 1, we have 

Xn,n ~ Xn,n O'lT'd ^n i = Xn^l for n < 771, 

X*nn> Xn^n O-'nd a^n 1 < f^''" U > m. 

Proof. We deduce Theorem 13 . 1 1 from a general result on the monotonicity of the extremal 
zeros based on the Hellman-Feynman theorem (see [11| . [211 Section 7.3 and 7.4] and the 
references therein). This general result states that if the coefficients a„ and /3„ are 
differentiable monotone increasing functions of the parameter A, then also the largest 
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root is an increasing function of A. In our case, the derivatives of the coefficients a„ and 
f3n with respect to the dilation parameter A are given by 

a^(A) = 0, n G No, 

/3;(A) = 0, n / m, /S^^) = > 

and therefore nonnegative. Thus, by Theorem 1.1] the largest root x* „ of P^ix) 
is a monotonic increasing function of A. For n < m, equation (jl6p implies that the 
polynomials P„ and P* and therefore also the zeros Xfi^n cind. ^ coincide. For n > tyi^ 
the Hellman-Feynman theorem impUes the formula (see |1H formula (2)], [21] formula 
(7.3.8)]) 



Ht* R P* (t* ^ P* 

^•^n,n h'm^ m\-^n,n)^ m—l\'^n,nJ „ 

for the derivative of x* „ with respect to the parameter A, where „ > denotes the 
Christoffel number corresponding to the zero x* So, for n > m and A > 1, the 
root X* „ is strictly larger than Xn,n- By the symmetry of the polynomials P*, we have 
X* = — X* This implies the statement for the smallest roots. □ 

Remark 3.2. Alternatively, it is also possible to prove Theorem 13.11 with the Perron- 
Frobenius theory. One way to do this consists in adopting |2H Theorem 7.4.1] to the 
setting of Theorem 13. II For the case m = 1, even stronger statements can be shown. For 
A > 1, Slim proved in [33] the following interlacing properties for the zeros x* and x„j: 



n 
12 



^n,j < ^n,j < x'^j-^l, for 1 < J < 

< j < n 



2 



In order to get residual polynomials that are small in the interior of [0, 1], it is important 
that all the zeros of the co-dilated polynomials P* are in the interior of the interval 
[— 1, 1]. Restricting the dilation parameter A appropriately, this can indeed be proven. 

Lemma 3.3. All the zeros of the polynomials P*(x), n E N, are in the interior of [—1, 1], 
if and only if 

A < (17) 

with the constant Lm given by 

Pnil) 1 
< Lm. ■■= lim „ < 1. 

n—m\ J 

Proof. By induction we prove the following identity for the numerator polynomials Pn'"^ : 
P„+i(x)Pi':i(x) - Pi"!l+^ix)Pn{x) = ' ' ' /3„P^_i(x), n>m. (18) 
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For n = m, this is clearly the recurrence formula for Pm+i- Assuming that (jlSp holds 
for an integer n > m, we show that (fT8|l holds also for n + 1. To this end we adopt the 
three-term recurrence formulas (|13p and (jlSp and get: 



Pn+2{x)Pi"ll,+l{x) - P!l"!:l^^^{x)Pn+l{x) 

= x{Pn+,{x)Ptl+i{x) - Pi!^i,+i(x)P„+i(x)) 

- Pn+l{Pn{x)Ptl+M - Ptl(.x)Pn+l{x)) 
= — /3m/3m+l • • • Pnl^n+lPm-lix) ■ 

For x > 1, we have Pm{x) > and P^l^(x) > for all n > m > 0. Then, using (|18p . 
we get the following chain of inequalities for x > 1: 

r, ^ Pn+l{x) _ Pn{x) PmPm+l ' ' lix) 

pM ( \ ~ pM ( \ pM (r)p'-"^^ (t) 

< f"P^ < • • • < = Prnix). (19) 

This implies first of all that < < 1 exists. Further, by the identity (jl6p we get 



P*{1) _ ^ Prrjl) ^ ^ ^ f >1 ifA<0, 



p!rlii)Pm{i) ptlii)PM I > - 1) + 1 if ^ > 0- 

Therefore, if A < jTrj— , then -P^(l) > and P^ix) does not change sign for x > 1 for all 
n > 1. On the other hand, if A > , there exists an n € N such that P^(l) < 0. Since 

the polynomials -P^(l) are monic, this implies that there exists a root of P^ix) larger 
than 1. Since P*{—x) = (— l)"P^(x), the respective statements hold also for x < —1. □ 

Remark 3.4. Chihara proved in [1] similar results for families of co-recursive orthogonal 
polynomials. The statement and the proof of Lemma 13.31 are adaptions of [H Theorem 
2] to the case of co-dilated polynomials. The case m = 1 of Lemma [3.31 is proven by Slim 
in |33) . For m = 1 the formula (|18p is also well-known, see [5l Equation 4.4]. 

The next Lemma shows that the critical point A = — in Lemma [3.31 is also a critical 
point for the asymptotic behavior of the normalizing factor P^i^)- 

Lemma 3.5. Let Lm > and A < j^^—- The sequence P^(l)/P„(l), n G is 
monotonically decreasing and its limit is given by 

p* ('1 N 1 — A 

lim^ = A + -— >0. (20) 
^/ A = then lim„^oo = 0. 
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4 Semi-iterative methods based on co-dilated ultraspherical polynomials 



Proof. Using formula p6|) . we obtain 



Pn{l) Pn{l) Pn{l) ' ' i^n(l) 



By p9p . the sequence ^" "p^^^"^^^^ is monotonically increasing and converges to j^. 
P*(i) 

Therefore, "p^jj is a monotonicahy decreasing sequence and for the hmit n — >■ oo we get 
1™ -B-JTT = A + lim — = A + — — =: /(A). 



-'m 



Since < Lm < 1, the function / is a strictly monotone decreasing in the variable A. 
Moreover, we have f{j^j^) = 0. This implies the statement of Lemma 13.51 □ 



4 Semi-iterative methods based on co-dilated ultraspherical 
polynomials 



As a main example of accelerated Landweber methods based on co-dilated orthogonal 
polynomials, we consider the ultraspherical polynomials Pn'^\ u > and its co-dilated 

relatives Pn'^* . The orthogonality weight function of the ultraspherical polynomials on 
[— 1, 1] is given by the function Wy{x) = (1 — x^Y'^i with the mass 

/3o= / w^,{x)dx= / {l-x^Y-2dx- ^ 2^ 



T{2u + l) 

The coefficients of the three-term recurrence relation ()13p can be written explicitly as 
(see da p. 29]) 

n(n -|- 2z^ — 1) , , 

= A( \ ^( ^ IV " ^ 21 

4(n + vj^n + V — 1) 

In this section, we will only consider co-dilated polynomials in which the first coefficient 
/3i is altered, i.e. in which m = 1 holds. To simplify the notation we will use the symbol 
Qn"^ to denote the first order numerator polynomials of Pji^K They can be represented 
as (see O Chapter III, formula (4.6)]) 

Q(:\x) := pH(^)(x) = 1 /" " f-^'^^^ w^iOdL nGNo. (22) 

We are using this representation to compute the critical value Li. 
Lemma 4.1. For u > ^, the quotient Q^}_^{1) / pji^^l) is given by 

Q^:Ul) _ 2u r(2^)r(n + i) \ ^23) 



21/- iV r(n + 2z.) 
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4 Semi-iterative methods based on co-dilated ultraspherical polynomials 



For — ^ < < ^, the sequence Q^^-^{1) / Pn'^\l) , n S N, diverges. Therefore, for the 
ultraspherical polynomials Pn\x), the constant Li in Lemma \3.3\ is given by 



2u-l -f 1 



r, = J 2u if ^<v, 

and the statement of Lemma \3.3\ holds for all A < max{l, 2z/}. 



Proof. For u > let < e < u + ^. Then, the function (1 + xY 2(1 — xj" 2 - is 
integrable on [—1, 1] and we have (cf. |211 equation (4.02)]) 



More generally, using the Rodriguez formula |34t (4.7.12)] for the ultraspherical polyno- 
mials Pn'^\ n > 0, we get the following integral formula 

^ pi''\x){i-xY~K (-i)"r(z^ + ^) ( d^"" 



-dx 



{i-x'y-2- 



-dx. 



-1 Pi,^\l){l-xY 2^T{n + u + \) j^i\dx) \{l-xY 

Integration by parts of the right hand side yields (using, as in equation (j24p . (4.02)]) 



1 P^")(l)(l-x)- 2"r(n + z. + i) y_i' (l-x)-+=' 

r{n + e)T{i^ + l) 22'^-=+"T(n + z. + i)r(z.-e + i) 



2"r(e)r(n + 1/ + i) r(n + 2i/-e + l) 

(25) 



2'^^~^r{u + i)r(i/ - e + i)r(n + e) 



T{e)T{n + 2u-e + l) 

Now, if z/ > ^, we can choose e = 1 and formula (|22|) for the numerator polynomials in 
combination with (fM|) and (|25p gives 



pi'^^(i) ^oJ^i\ pM(i); i-e 

_ 2^'-ir(z/ + l)T{u - i) 2^'-"ir(z. + ^)T{u - ^)r(n + 1) 
/3or(2z/) /3or(n + 2z.) 

2u r(2z^)r(n + l) 



2i/ - 1 V r(n + 2v) 
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4 Semi-iterative methods based on co-dilated ultraspherical polynomials 



On the other hand, if — i < < i, we choose e < + i and get with (1241) and ([25 



> 



/3or(2z^ - e + 1) 
r(i/ + i-e) 



1 



r(2i/-e + l)r(n + e) 
r(e)r(n + 2z^ - e + 1) 



T{n + e) 



2/3q r(n + 2i/ - e + 1) 

Therefore, we can find an G N such that 

Qil(l) ^ r(z. + i-e) 



> 



for ah n > Ui;. 



Since e can be chosen arbitrarily close to u + ^, the term on the right hand side can 
be arbitrarily large. Hence, in this case the sequence Q'^}_^{1)/ Pn'^\l), n G N, diverges. 



The formulas for the constant Li follow from the definition Li = lim. 



n->oo -TU) 



□ 



Lemma 4.2. For v > ^, the polynomials Pn^* {x) / Pn'^* {!) are uniformly hounded on 
[-1,1] hy 



Pi.'^*{x) 



p(^)*(l) 



< 



\2u(2\~l)-\\ 
2u-\ 



if < A < 1, 

if l< \ <2v or A < 0. 



(26) 



Proof. For the ultraspherical polynomials with the parameter > 0, it is well-known (see 
[55] and the references therein) that \Pi''\x)\ < P^^^l) and \Qli^\x)\ < Q^n\l) holds 
for all X S [— 1, 1], n S N. In the case < A < 1, we get therefore by formula (|16p the 
upper bound 



Pt^*{x) 




XPt\x) + (1 - X)xQ^^l,{x) 


Pi'^^il) 




XP^\l) + {1 - X)Q^:Ul) 



^ xp^''\i) + {I - x)Q':i,{i) _ ^ 



xp^\i) + {1 - x)Q):i,{i) 



Similarly, we get for 1 < A < 2i^ or A < 0: 



Pt^*{x) 



pt\x) + ^Q):i,{x) 



< 



< 



^'i'^^(i)-^Qi':^i(i) 



Li H ^ 



2u~l 
2y 



A 



2u-l 
2u 



+ 



1-A 



In the last inequality, we used the fact that Li 



p^'\i)+'-^Q^:l{i) 

_ \2u{2X - 1) - A| 
~ 2v-X ■ 



2u-l 
2u 



lim 



for all n € N (see formula ()19p in the proof of Lemma l3.3p . 



< 



holds 



□ 
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4 Semi-iterative methods based on co-dilated ultraspherical polynomials 



The weight function wi^^ of the numerator polynomials Qn^ is supported on the interval 
[— 1, 1] and can be stated explicitely as (see |15| formulas (28) and (106)]) 

wi'\x) := - x'y-K <^Ax):= \ (27) 

Po 

Jo {i-t^Y+^ 

We will soon see that for X <2v and x g] — 1, 1[, the normalized co-dilated ultraspherical 
polynomials Pn^* [x) / Pn^* {1) converge pointwise to zero. The proof is based on the fact 
that for I' > \ the weight function is a generalized Jacobi weight (see |29l Definition 

9.28]), i.e. is of the form ()27p with a continuous and strictly positive function ^u{x) 
on [—1, 1] whose modulus of continuity uj satisfies 5~'^u){(ty, 5)d5 < oo. 

Lemma 4.3. For u > ^, the weight function wl^^ is a generalized Jacobi weight. 

Proof. We show that for v > ^ the function is strictly positive and Holder-continuous 
on [—1, 1]. Then, it follows immediately that Jq 5~^uj{^^, 5)d6 < oo holds and, thus, that 

w^'^ is a generalized Jacobi weight. Clearly, the weight function Wuix) = (1 — x'^Y~^ of 
the ultraspherical polynomials is Holder-continuous on [—1, 1] with exponent min{l, — 
^}. The function q^, on the other hand is continuously differentiable on the open interval 
(— 1, 1). So, to complete the proof it remains to show that Qi, satisfies a Holder-condition 
and is nonzero at x = 1 and x = —1. Because of the symmetry of the weight function 
wl^\ we have to study the behavior of q,/{x) only at x = 1. To investigate the integral 
formula, we proceed similar as Szego in \34:\ Section 4.62] for the Jacobi polynomials of 
the second kind. We expand the factor (1 -|- tY^^ in the integral formula (|28p of qu{x) 
in a power series in the variable (1 — t). Then, if v > ^ and u — ^ is not an integer we 
obtain 

q,{x) = C(l - x2)--| + Ml (^) (1 + xY~K 



with a power series Mi{y) convergent for \y\ < 1 and Afi(O) 7^ 0. Thus, in this case 
the function qi, is nonzero at x = 1 and satisfies a Holder-condition with exponent 
min{i/ — ^, 1}. If n = — ^ > 1 is an integer, we get in the integrand of ()28p the power 
series expansion (see |34| p. 76]) 



(l-t2)n+l 2*^+1(1 -t)"+l 2"+l(l -t)'^+l V 

Integrating with respect to t yields a logarithmic term and a power series M2{y) with 
-^2(0) 7^ converging for |y| < 1 such that 

q,{x) = C(l - x^Y log {-r^\ + M2 (^) (1 + xY- 
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4 Semi-iterative methods based on co-dilated ultraspherical polynomials 



Thus, in this case qu{x) is Lipschitz-continuous at x = 1 if n > 2, and Holder-continuous 
with an arbitrary coefficient 0<a<lif?i=l. □ 

Theorem 4.4. For u > o.'^d X < 2v, the co-dilated ultraspherical polynomials Pn^* 
satisfy the estimate 

i + |A| a 



< 



with a constant Cy independent of n, x and A. For the respective residual polynomials 
Tn''*{y) = ^" fSl on [0, 1] and the modulus of convergence e^{n), we get 



(y) 



(1) 



< 



1 + |A| 



4 in) < 



{2u-\) (2y^(r^n + l)-' 



1 + |A| 

{2v - A) {2n)'- 



Proof. Since both, Wy and are generalized Jacobi weights, we get for Pn '^ and Qn^ 
the uniform bounds (see [1] Lemma 1.3] or |29| Lemma 9.29]): 

n iipHli V n \ n ||p(^)|| 



with constants Ci and C2 independent of x G [—1, 1] and n G N. Due to the particular 
normalization ()27p of the weight function the weighted L^-norms of the monic 

polynomials Pn^ and Q^^-i coincide (see [15l Section 2]), i.e. HPn'^^H^;^ = HQlj'^ij^H (i). 
This yields the following estimate for the co-dilated polynomials: 



IV |Pi")*(x)| / r . , l^' 



11^^"^ (^) 



X^ + 



APi'^)(x) ^(l-A)xQi^2i(x) 



< \X\Ci + |1 - AIC2 < (1 + |A|)(Ci + C2). 



By Lemma [3. 51 and Lemma l4.ll we have ^^l^y^ < = Therefore, 

y^-^ ly l-Pn*^^*!^)! < "^f ^"-- ^^-^ (1 + |A|)(Ci + C2). (29) 
"7 P^")*(l) Pi")(l) 2i/-A^ ' '^^ 

For the ultraspherical polynomials, the quotient Pn'^\l)'^ /\\Pn'^^\\'^^ can be computed 
explicitely as (for the formulas, see |13l p. 30]) 

Pi"\lf ^ rjn + 2u) n + u 

||pH||2 r(n + l) 22-ir(i.+ i)2- ^ ^ 
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4 Semi-iterative methods based on co-dilated ultraspherical polynomials 



Using two inequalities related to the formula of Gosper for the Gamma function T{x) 
(see |27| Theorem 1]), we get for > ^ the following lower bound: 



n+2u-l \n+2u~l /^^ , ^ , i 



||pH||2^^ - (f)"V2m 22-ir(z. + i)2 - (2ep-ir(z. + i)2- 

Now, including this inequality in the estimate ()29p . we get a constant Ci, independent of 
X <2iy, X e [-1, 1] and n G N such that 



\ n 



^ (l + |A|) a 



The estimates for the residual polynomial rn^* on [0, 1] and the modulus £y{n) follow 
directly from the respective definitions. □ 

Remark 4.5. Theorem 14.41 implies that for the semi-iterative algorithms based on the co- 
dilated ultraspherical polynomials Pn^* with parameter A < 2z/ the symmetric modulus 
of convergence is of order ef(n) = 0(n~*) for < s < z^. Therefore, according to |16| 
Theorem 4.1]), the co-dilated ultraspherical polynomials provide a semi- iterative method 
with optimal order 0{n~'^) of convergence if the solution / is an element oiXg, {) < s < v. 

Finally, for the co-dilated ultraspherical polynomials, we compute the coefficients ^n+i 
in Algorithm [1] explicitly. To this end, we need first of all an explicit formula for the 
quotient Pn\'\^) / Pn^i{^) ■ We obtain this quotient by using formula (j30p and the fact 
that ||-P^![_\ llio^ = Pn+i \\Pn'' Ww^ holds for the monic polynomials Pn ''- Thus, we get 

P^'\l) _ P^'\l) llPi^ilU .n - 



2- 



Now, using formula (|23p . we get the coefficients fin+i, n> 1, explicitly. 



^Sd) ^ii(i)A+(i-A)gs;m 



n + 2iyx,(-, 2u (. T{2v)V{n+2) \ ' ^ 

With a simplified expression for Hn+i, the semi- iterative method based on the co-dilated 
ultraspherical polynomials is summarized in Algorithm [2j 
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5 Co-dilated v-methods 



Algorithm 2 Semi-iterative metliod based on co-dilated ultrasplierical polynomials 
/o = 0, /i = 2 coA*g 

while (stopping criterion false) do 

- I (2^-A)r(«+2i.)+(A-i)r(2i.+i)r(«+i) 

^tji+l — -I- 1^) (2jy-A)r(n.+2i/+l)+(A-l)r(2iy+l)r(n+2) 

fn+1 = fn + (/^n+l " l)(/n " fn-l) + 2^„+l UjA*{g - Afn) 

n ^ n + 1 
end while 



5 Co-dilated z/-methods 



The i^- methods correspond to Algorithm [T] with the recurrence coefficients a„, /3„ of the 
monic Jacobi polynomials pjl' ^' on [—1, 1]. These particular orthogonal polynomials 
are linked to the ultraspherical polynomials Pn'^^ by the formula (see |34| Theorem 4.1], 
using the normalization of the monic polynomials) 



In other words, the polynomials P„ ^ ' ^ describe the positive part of the axisymmetric 
ultraspherical polynomials P2n ■ Thus, for the asymmetric residual polynomials '^n^ of 
the i/-methods, we have 



Compared to the semi-iterative methods based on the ultraspherical polynomials, the u- 
methods have the advantage to converge if a;|| A*j4|| = 1. A similar approach for arbitrary 
symmetric orthogonal polynomials leads us now to semi-iterative methods that generalize 
the z^-methods. 

In general, if P2nix) is an arbitrary even polynomial of degree 2n on the interval [—1,1], 
then P2n(\/1 — y) defines a polynomial of degree n in the variable y on the interval [0, 1]. 
In this case, we can define asymmetric residual polynomials by 



^n(?/) := ^^44^, ye [0,1]. (32) 



Moreover, if the symmetric polynomials P„ satisfy the three-term recurrence formula (fT3|) . 
we can deduce directly a three-term recurrence relation for the residual polynomials ^ri- 
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5 Co-dilated v-methods 



Applying the relation (jl3p twice, we get first for the even polynomials the recurrence 

P2n+2{x) = (X^ - /32„ - /32n+l)P2n{x) - /32n/32n-l-P2n-2 (x) , n > 1, 

Po{x) = l, P2{x) = x^-Pi. (33) 

Inserting (j33p in the definition ()32p , yields the following recursion formula for the residual 
polynomial 'V„(y) on [0, 1]: 

\n+l{y) = (1 - ?/ - /32n - ,92n+l)7^^^ - /^a A-i ^ (y), 

%(y) = 1, = (34) 

i - Pi 

By the formula ([33|) . also the factors '^//n+i = p^^"2'(i) computed recursively. This 

results in the following semi- iterative Algorithm ^ 

Algorithm 3 S emi- iterative method based on the asymmetric residual polynomials 

a, , 1 

^1 



/o = 0, /i = 

while (stopping criterion false) do 



/n+1 = /n + ((1 - hn - P2n+l) X+1 " l)(/n " /n~l) + X+1 W^*(5 " ^/n) 

n — 7- n + 1 
end while 



In the light of ()32p . we can introduce asymmetric residual polynomials also for the co- 
dilated orthogonal polynomials P* by setting 

''<{y):=^^^'^{^^. ye [0,1]. (35) 



In view of the recurrence relation (|14p of the co-dilated polynomials P*, the residual poly- 
nomials 'V* satisfy the same recurrence relation (j34p as the polynomials ^V^ except that 
the two coefficients including Prn are altered. Families of orthogonal polynomials in which 
more than one coefficient is altered are known as co-modified orthogonal polynomials. 
As the co-dilated polynomials, they are well studied in the literature, see [8l 1261 [3T]. 

From Theorem 13.11 and the Lemmas 13.31 and 13.51 we can moreover deduce the following 
results about the zeros of the polynomials The statements follow directly from the 
relation (j35p of the polynomials ^* to the polynomials Pjn- 

Corollary 5.1. The smallest zero of^^ is a decreasing function of the dilation parameter 
A. All zeros of^^{y), n G N, are in the interior of [0, 1] if and only if X < . 
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5 Co-dilated v-methods 



Finally, for m = 1, we consider in more detail the asymmetric residual polynomials 
^n^*{y) = P2n*iV^ ~ y) / ^2ti* i'^) linked to the co-dilated ultraspherical polynomials. 
As a consequence of Theorem I4.4| we get the following estimates for 

Corollary 5.2. For m = 1, u > ^ and A < 2v, the residual polynomials ^n^*{y) on 
[0, 1] and the modulus of convergence £u{n) are hounded by 



{2u - A) {2^n + 1)^ ' ' ' - (2i/ - A) {2n)^ ' 

The constant is independent of n, y and A. 

Remark 5.3. For A = 1, the result of Corollary 15.21 corresponds precisely to the well- 
known convergence result of the i/-methods, see |im Theorem 6.12]. In Corollary 15. 2[ 
the residual polynomials '^n^* converge pointwise to zero at y = 1. Thus, compared to 
the symmetric polynomials rn^* of Theorem 14.41 the residual polynomials ^^^* define 
semi-iteratiye methods that converge also to zero if a;||A*^|| = 1. 

Remark 5.4. The convergence orders £u{n) = 0{n~''^) obtained in Corollary 15.21 are 
substantial for the usage of the co-dilated z^-methods as regularization methods. In 
particular, |10l Theorem 6.11] implies that the co-dilated i/- method with \ < 2u based 
on the residual polynomials '^n^* is a regularization method of optimal order for / G Xg 
with 0<s<i^ — lif the iteration /„ is stopped according to the discrepancy principle, 
i.e. if \\Afn — g\\ < re. Here, e denotes the noise level of the data and the parameter r 
is chosen larger than the uniform bound supj^gjg | ^n'^*{y)\ given in Lemma [4.21 Using 
a generalized discrepancy principle as stopping rule, as described in |10| Algorithm 6.17] 
and |18) . the co-dilated i^-methods even provide an order optimal regularization method 
for < s < v (see \10\ Theorem 6.18]). 

In Algorithm [3l the coefficients ^^n+i for the co-dilated ultraspherical polynomials are 
given explicitly as ^fin+i = fJ'2n+if^2n+2, the factors /in+i given in (|3ip . With the recursion 
coefficients of the ultraspherical polynomials given in ()2ip . we summarize the co- 
dilated i^-methods in Algorithm [H 

Algorithm 4 Co-dilated z^-methods 

/o = 0, /i = u:A*g 

while (stopping criterion false) do 

^1 - 4f2n + u)(2n + u + I) (2'--A)r(2n+2i.)+(A-i)r(2^.+i)r(2n+i) 

/n+1 = fn + ((1 - 2(2n:l+Vn+"J-l)) '/^"+l " ^ (/« " fn-l) + X+1 ^A*{g - Af,) 

n — 7- n -|- 1 
end while 
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6 Adaptive choice of the dilation parameter A for the co-dilated 1-method 



6 Adaptive choice of the dilation parameter A for the 
co-dilated 1-method 



In the algorithms of the last sections it is a priori not clear how the dilation parameter 
A has to be chosen. In the following, we provide for the co-dilated 1-method a simple 
adaptive scheme that computes for every step n of the iteration an optimal A such that 
the error ||A/„— g|| is minimized. For v = 1, the ultraspherical polynomials Pn'^ coincide 
with the Chebyshev polynomials Un of the second kind. This yields in Algorithm U] the 
coefficients 

a„ ^ ^ln(l) ^ (2-A)2n + A 
'^"^^ ^2n+2(l) (2-A)2n + 4-A' 
resulting in the iterative scheme 

„ ^ (2- A)2n + 3A-4 , „ „ , ^ (2 - A)2n + A 
- + (2_A)2n + 4-A " ^""^^ + ^ (2 - A)2n + 4 - A ^ " 

/i = A*g, fo = 0. (36) 

For A = 1, this iteration is precisely the Chebyshev method of Nemirovskii and Polyak 
(see |30| p. 150]). Due to the particular three-term recurrence formula ()12p of the 
Chebyshev polynomials C/„, it is possible to calculate the iterates /„, for all different A 
at one stroke. Namely, by the last identity in equation ([8]) the residual polynomials 
*¥*(?/) := ^n^*{y) can be written as 



^2n(l) U2n{VT^) 1 - A ^2n-2 (1) ^2n-2 (^/T^) 
C/|„(l) C/2„(l) 4 [/|„(1) t/2n-2(l) 

^^^^ ^r.iy)^ ^]-'}iT-l^ ^r.^.iy). (37) 



(2-A)2n + A '''' (2-A)2n + A 

In this way, every residual polynomial (y) can be computed as an affine combination of 
the residual polynomials and ^n-i- This enables us to introduce a low-cost adaptive 
algorithm in which in every step n the parameter A is chosen optimally. If denotes 
the iterate in ()36p with respect to a fixed parameter A G M, we have 

min \\Af}: - g\\ = min \\Aif^ - ^[fn " /n-i)) " ol 

7gM 



The minimum on the right hand side is obtained if the vector — 7(/^ — fn-i) is 
orthogonal to — i-e. if 

W A / rl / i!l rl \\ II (/n) fn fn—l) 

argmin||A(/„ - 7(/„ - /„_i)) - g\\ = -rji ti — 

\\fn-fn-l\\ 
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7 Numerical tests 



Thus, in view of (j37p . the optimal A after n steps of the iteration (j36p is given by 

. . (2n +!)(/!, /^ -/^i) 

(2n-l)(||/i-/i_J|2- (/I, /I 

This simple idea is summarized in the adaptive co-dilated 1-method formulated in Al- 
gorithm [5l Here, the iteration is stopped according to the discrepancy principle if the 
minimal error min;^g]R \\Af^ ~ 9\\ gets smaller than re, where r > 1 and s describes the 
noise level of the data. 



Algorithm 5 Adaptive co-dilated 1-method 



/o = 0, /i = lujA*g 




Vo = g, vi = g - Afi 

(vi,vi—vo) 


- 7(z;i - Vo) 


while fmin > T£ do 




fn+l — /n + 2n+3^f^ 


-fn-l)+^U;'^S^*Vn 


Vn+1 = g- Afn+1 




_ (l'n+l,fn + l— fn) 

' ~ ||t)„+l-j;„|P 




Vmin = Vn+l - l{Vn+l " 




n — )• n -|- 1 




end while 




X _ 1 (2n+l)7 
- (2n-l)(l-7) 




fn — fn 'yifn /n— l) 





Remark 6.1. Similar adaptive schemes are in principle possible also for the other co- 
dilated i/-methods. Taking two arbitrary real values Ai 7^ A2, every iterate f^ can be 
written as an affine combination of f^^ and f^^. Thus, in order to obtain all iterates f^, 
it suffices to compute two iterates f^^ and f^^. However, for general u there exists no 
direct relation between f^^ and f^^ such as in the case of the Chebyshev polynomials. 
Therefore, for i' ^ 1 the iterations in adaptive schemes like Algorithm [5] are twice as 
expensive as in Algorithm 21 



7 Numerical tests 



In this final section, we compare the convergence behavior of the co-dilated i^-methods 
with the original i/-methods and the Landweber method. As a first and very simple test 
equation we consider 

Af = gl, A = diagn,^,---^j , gl = eN + ew, (38) 
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7 Numerical tests 



with Cat = (0, • • • ,0, 1), e > and w a vector of normally distributed Gaussian white 
noise. To solve ()38p . we use Algorithm 21 We choose cj = 1, e = 0.01, = 100 and 
stop the iteration according to the discrepancy principle, if \\Afn — || < 4e. For v = 1 
the number of necessary iterations depending on the dilation parameter 1 < A < 2.2 
is given in Figure [2l For v = 2, the number of necessary iterations depending on the 
parameter 3.9 < A < 4.05 is depicted in Figure [31 In both figures, the smallest zeros of 
the respective residual polynomials are plotted on the right hand side. 



Figure 2: Convergence of the co-dilated 1-method (Algorithmic with = 1) to solve ([3 
depending on the dilation parameter A. 



Number of iterations to solve ([38]) with Smallest zero of the residual polynomial 
Algorithm |4] with = 1 depending on '^150* (y) the interval [0, 1] depending on 
1 < A < 2.2. 1 < A < 2.2. 



Figure 3: Convergence of the co-dilated 2-method (Algorithm U] with v = 2) to solve ([3 
depending on the dilation parameter A, . 



1 

0.8 
0.6 
0.4 



S.9 3.95 4 4 

Number of iterations to solve (|38p with 
Algorithm |4] with = 2 depending on 
3.9 < A < 4.05. 



9 3.95 4 4.05 

Smallest zero of the residual polynomial 
^215* (y) in the interval [0, 1] depending on 
3.9 < A < 4.05. 
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The graphs in Figure [2] and [3] indicate that for the hnear system ()38p it is favorable to 
choose the dilation value A in Algorithm U] close to but smaller than the critical value 2z^. 
This fact can be explained by the particular structure of the matrix A and the right hand 
side gf. The vector is up to a small perturbation exactly the eigenvector of A* A with 
respect to the smallest eigenvalue 0.0001. For a fast termination of Algorithm 3] it is thus 
favorable if the residual polynomials 'h'n^* decay fast at zero. This is guaranteed if the 
dilation parameter A is close to the critical value 21/. For v = 1, the adaptive Algorithm 
[5] stops after n = 95 iterations and gives the optimal parameter A = 1.9930696. 

Figures [2] and [H] also indicate that the number of iterations to solve (|38|) is strongly linked 
to the smallest zero of the residual polynomials '\n^* . The smallest zero of the residual 
polynomial ^n'^* in [0, 1] is a decaying function of the parameter A until the critical 
value A = 2i/ is attained. At the critical value A = 2i^ we cannot expect convergence of 
Algorithm m and this is also verified in Figures [2] and [3l For values of A larger than 2v the 
smallest root of '^n^* is for large n strictly less than zero. In this case the second smallest 
root of ^n^* is the smallest root in the interval [0, 1]. The convergence of Algorithm U] 
is now linked to the position of the second smallest root of ^n^* . 

After having considered a good-natured example, we give a second example in which 
Algorithm U] does not improve if the parameter A is increased. We consider as a second 
test equation 

Af = gl gl = e2 + ew, (39) 

with A and w given in (|38p and 62 = (0, 1,0, ••• ,0). Again, we use Algorithm H] with 
LO = 1, e = 0.01, = 100 to solve (|39|) and stop the algorithm if the error \\Afn — ^fll 
is less than 4e. In this second case, the vector is up to a small perturbation the 
eigenvector of A* A with respect to the second largest eigenvalue 0.25. Here, we can 
not expect that residual polynomials with a fast decay at the origin will have a strong 
effect on the number of iterations in Algorithm HI The diagrams in Figure [J] confirm this 
expectation. The adaptive Algorithm [5] for 1^ = 1 stops in this case after n = 65 iteration 
with the optimal parameter A = 1.6003658. 
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Figure 4: Convergence of Algorithm |4] to solve ()39p depending on the parameter A. 
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Number of iterations to solve ()39p with 
Algorithm U] with v = 1 depending on 
< A < 2.2. 




3 3.2 3.4 3.5 3.8 ^4 

Number of iterations to solve ()39p with 
Algorithm 3] with v = 2 depending on 
3 < A < 4.2. 



As a final example of an ill-posed linear problem, we consider the well-known Fredliolm 
integral equation of the first kind (see [6l Example 12.4.1.] 

Af{s) = r k{s, t)f{t)dt = g{s), < s < 1, 
Jo 

with the right hand side g{s) and the kernel k{s,t) given by 

— s 

g{s) = — ^ — , k{s,t) = 



6 



t{s-l) < t < s < 1, 
s{t-l) < s < t < 1. 



The exact solution of this equation is given by the second derivative g"{s) = s. A 
discretized version of this integral equation based on the Galerkin method is included 
as test problem deriv2 in the regularization toolbox of Hansen |19| . As a number of 
discretization points in deriv2 we choose N = 50. Further, as in the previous test 
examples we disturb the right hand side 5 by a vector ew with e = 0.01. In Algorithm 
m we choose w = 96.5 to guarantee a;||A*A|| < 1 and stop the iteration according to the 
discrepancy principle if \\Afn — g^\\ < 4e is satisfied. The different numbers of iterations 
in Algorithm U] depending on the dilation parameter A are illustrated in Figure [5] and 
Tabled! 
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Figure 5: Number of iteration steps of the co-dilated i/-method to solve the test problem 
deriv2 in the toolbox of Hansen 1191 




1 1.2 1.4 1.6 1.8 2 2.2 T99 3.992 3.994 3.996 3.998 4 4.002 4.004 4.006 4.008 4.01 

Number of iteration steps of the co-dilated Number of iteration steps of the co-dilated 
1-method, depending on 1 < A < 2.2. 2-method, depending on 3.99 < A < 4.01. 



Table 1: Convergence of different semi- iterative methods to solve the test problem deriv2 



Semi-iterative method 


Iteration steps 


Semi-iterative method 


Iteration steps 


= 1, A = 


1007 


1/ = 2, A = 


1290 


= 1, A = 0.5 


1007 


1/ = 2, A = 0.5 


1290 


= 1, A = 1 

1-mcthod (Ncmirovskii-Polyak) 


1006 


1/ = 2, A = 1 

2-method 


1290 


zy = 1, A = 1.5 


1005 


i/ = 2, A = 3.9 


1290 


zy = 1, A = 1.9 


998 


u = 2, X = 3.99 


1289 


zy = 1, A = 1.99 


932 


u = 2, X = 3.999 


1280 


zy = 1, A = 1.99716 

Adaptive Algorithmic (optimal) 

z/= 1, A = 1.9999 


884 
1498 


1/ = 2, A = 3.9999 
1/ = 2, A = 3.99998 


1184 
886 


cg-method 


23 




Land web er 


359379 





Figure and Table [T] illustrate that for the test problem deriv2, similar as for equation 
()38p . the total number of iteration steps of the co-dilated i/- method gets significantly 
smaller if the parameter A approaches the critical value 2i/. Also, if A is too close to 2i/, 
we get very slow or no convergence of Algorithm For z/ = 1 , the adaptive Algorithm 
[5] of the previous section gives the optimal parameter A = 1.99716 after n = 884 steps. 

Further, Table [T]shows that the z/-methods and the co-dilated i^-methods are significantly 
faster than the Landweber method. On the other hand, it is also visible that the cg- 
method outperforms all semi-iterative methods in which the coefficients are a priori given. 
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For a further comparison between the performance of the z^-methods and the cg-iteration, 
we refer to |16j . 
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